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Abstract 

We rephrase the problem of 3D reconstruction from images in terms of inter- 
sections of projections of orbits of custom built Lie groups actions. We then use 
an algorithmic method based on moving frames a la Fels-Olver to obtain a funda- 
mental set of invariants of these groups actions. The invariants are used to define 
a set of equations to be solved by the points of the 3D object, providing a new 
technique for recovering 3D structure from motion. 

1 Introduction 

The concept of invariance is of major importance in modern geometry. In the field of 
computer vision, invariants have been used for more than a decade for object recogni- 
tion and reconstruction (see for example |[|). 

One particular vision problem one could hope to solve using invariants is that of 
reconstructing a 3D object from a set of pictures taken from unknown viewpoints. For 
example, given n ordered points Oi , O2 , ■ • • , On G K'^ (the object) and t sets of n 
ordered points pi , . . . , p„ € (for each of the pictures), we would like to determine 
the object in R'^ from the images. 

One possible way of solving this problem would be to define an equivalence rela- 
tion between all the possible pictures of an object and to find functions computed on 
the pictures that are constant on each equivalence class. Characteristics of the object 
could be inferred from these functions, regardless of the camera position relative to the 
object. To define such an equivalence class, one can try to use the orbits of a group 
action. Real valued functions that are constant on the orbits of a group action are called 
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invariants. We would thus be interested in finding invariants of a group action which 
would be transitive on the set of pictures of any given object, i.e. "view invariants". 

Unfortunately, as is commonly known in the vision community, view invariants 
do not exist for 3D point sets of arbitrary size (in general position). One can still 
build invariants for specific objects (for instance, planar sets of points, pencils of lines, 
etc.) but not for arbitrary shapes. The problem is that the set of pictures of an object 
intersects with the set of pictures of other objects. Observe that if a view invariant / 
takes a constant value c for all pictures of an object O, then / is also equal to c on the 
set of pictures of any object whose set of pictures intersect with the set of pictures of 
O. One can actually show that any equivalence relation between all the pictures of 
each objects defines a unique equivalence class on the space of pictures and thus, any 
view invariant is trivial. From a group point of view, this means that any group action 
that is transitive on the set of pictures of any object must be transitive on the set of all 
pictures. 

Another way to solve this problem would be to use the reverse approach: try to 
characterize all the possible objects corresponding to a given picture. It would be use- 
ful to find functions that are constant on equivalence classes that include all the objects 
corresponding to a given picture. However, it is easy to see that such equivalence 
classes of objects are in one to one correspondence with the equivalence classes of pic- 
tures discussed earlier, and thus that only one such equivalence class can exist. There 
are therefore no "object invariants". Nevertheless, given a view of an object, we can 
infer information about the object, so there must be a way to overcome this difficulty. 

It is the fact that we know how a camera builds images (using the perspective pro- 
jection) that will allow us to build an equivalence relation which characterizes objects 
corresponding to different pictures. The key is to consider an equivalence relation on a 
higher dimensional space, sort of lifting the set of objects of different pictures to differ- 
ent "heights" in the extra dimensions. We can do this using the three extra dimensions 
provided by the camera center position. More precisely, we can construct a Lie group 
action on the object points Oi, . . . , On and the camera center Pq which summarize 
what is unknown about the object-camera system given a picture of an object. Invari- 
ants of this group actions prove to be sufficient for solving the problem of recovering 
the object coordinates in M.^, i.e. the "structure from motion" problem. 

Recent advances in the theory of moving frames ^ provide us with a systematic 
way to proceed to obtain the invariants of any given (regular) Lie group action. Using 
this systematic method, we obtain the invariants of our custom built action without any 
difficulty. We can even venture a bit further by modifying the group action and repeat 
the exercise so to obtain even more new tools for 3D structure and motion analysis. 

We begin our exposition with a summary of some of the relevant theoretical as- 
pects of the theory of moving frames and its application to the computation of (joint) 
invariants. In Section 3, we formulate the problem of structure from motion in this 
framework and obtain the corresponding invariants. We end the section by remark- 
ing how this leads to a simple test to identify camera motions that are pure rotations. 
Section 4 presents some variations of our approach where more camera parameters are 
considered, including a generalization to the case of a variable focal distance. 
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2 Theoretical Foundations 



Let M be an m-dimensional smooth (Hausdorff) manifold and G be an r-dimensional 
Lie group acting (smoothly) on M. 

Definition 2.1. An invariant \s a function / : Af — » R which remains unchanged under 
the action of the group. In other words, 

I{g ■ z) I{z), for all z e M and all g e G. 

A local invariant is a function I : M M. for which there exists N^, a neighborhood 
of the identity e € G, such that 

I{g ■ z) = I{z), for all z e M and all g e N^. 

Definition 2.2. We say that G acts semi-regularly on M if all orbits have the same 
dimension. If, in addition, any point po € M is surrounded by an arbitrarily small 
neighborhood whose intersection with the orbit through po is connected, then we say 
that G acts regularly. 

The following theorem, due to Frobenius [|[|, is of central importance to our ap- 
proach. A proof can be found in [||]. It provides us with a simple way of characterizing 
the orbits using invariants. 

Theorem 2.3 (Frobenius Tlieorem). IfG acts on an open set O C M semi-regularly 
with s- dimensional orbits, then Vpo G O there exist m — s functionally independent 
local invariants /i, . . . , Im-s defined on a neighborhood U ofpo such that any other 
local invariant H defined near pg o function H = /(/i, . . . , Im-s)- U G acts 
regularly on O, then we can choose /i, . . . , Im^s to be global invariants on O. In 
that case, two points pi,P2 G O are in the same orbit relative to G if and only if 
Ii{pi) = Ii{p2),for all i = 1, . . . , m - s. 

"Qy functional independence of the (smooth) functions /i, . . . , Im-s on an open 
set O, we simply mean that the Jacobian matrix of /i, . . . . Im-s has maximal rank 
m — s on an open and dense subset of O. The set {/i, .... /,n-s} is often called a 
complete fundamental set of invariants on O. Note that the complete fundamental set 
of invariants is by no mean unique. 

As we shall consider actions on points, we are interested in the case where M = 
)/' X V X . . . X V (n-times) =: V^*^") is the Cartesian product of n copies of a manifold 
V. 

Definition 2.4. We say that G acts diagonally on V ^ if there exists an action of G 
on V such that for any g £ G, for any n £ N and any 2:1 , . . . , z„ £ V, the action 
g ■ {zi, . . . , z„) can be written as 

g- (zi,... ,z„) ^ {g*zi,... ,5*z„). 

The group actions we will define for our object-camera systems are not diagonal 
actions. However, we will find a normal subgroup H of G such that 
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1 . The subgroup H can be written as H — x . . . x Sj (n-times) where S)is a Lie 
group acting on V in a consistent manner with the action of H, i.e. {hi , . . . ,hn)- 
(zi, . . . ,Zn) = . . . , hn*Zn), for all hi,. . . ,hn& Sj andallzi, . . . , z„ G 
V. This insure that V ^ (" V-H^ can be written as V ^ V-H^ = C^/^l) ^"^ ■ 

2. The action of G/iJ on V^("V^= {V/S^Y^''^ is diagonal. 

So for all practical purposes, we shall ultimately have to deal with diagonal group 
actions. 

In our approach to structure from motion, invariants are used to obtained equations 
that must be satisfied by the object and the camera. The more invariants we have, the 
more equations need to be satisfied. We need enough equations to completely deter- 
mine the object. Observe that the dimension of the orbit is bounded by the dimension 
of the group. So in the case of a diagonal action, taking more and more copies of V 
(i.e. more and more points) allows for the existence of as many invariants as necessary. 
The question that remains is: how can we obtain an expression for these invariants? 
Thanks to recent advances in the theory of moving frames [|[ ^, this problem can 
now be solved in an algorithmic fashion. We now summarize some of this theory and 
show how moving frames can be used as a tool to obtain a complete set of fundamental 
invariants. 

Definition 2.5. A (right) moving frame is a map p : M ^ G which is (right) equivari- 
ant, i.e. p{g ■ z) = p{z)g^^, for all 5 G G and z G M. 

Unfortunately moving frames do not exist for all group actions. 

Tlieorem 2.6. A moving frame exists if and only if the action if the group action satis- 
fies 

{g G G\3z G M,g ■ z = z} = e, 
where e denotes the identity in G. This property is called freeness of the group action. 

Demanding freeness of the group action is very strong. It appears that, in order to 
be able to deal with the generic cases, we need to relax this condition a little bit. 

Definition 2.7. A local moving frame is a map p : M ^ G such that p{g ■ z) = 
p{z)g~^, for all g G N^, a neighborhood of the identity e G G, and all z G M. 

Tlieorem 2.8. A local moving frame exists if and only if there exists a neighborhood 
Ne of the identity ;n e G G such that 

{g G Ne\3z e M,g ■ z ^ z} = e, 

or equivalently, if and only if for all z G M, the dimension of the orbit through z is 
equal to r, the dimension of G. This property is called local freeness of the group 
action. 
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Our hope is that, by acting diagonally on more and more copies of a manifold, 
we shall eventually obtain a locally free action. So we are interested in determining a 
simple criterion on the group action of G on V to determine whether or not this will 
work out. 

Definition 2.9. We say that G acts on M effectively if 

{.g e G| g -p^p, for all p e M} = {e}. 

We say that G acts on M locally effectively if 

{.g e G| g ■ p = p, for all p G M} is a discrete subgroup of G. 

Many groups do not act effectively. However, given G acting non effectively on 
M, we can consider G = G/Gm, where Gm = {g & G\g • z = z, Vz e A/}, which 
acts in essentially the same way as G except that it acts effectively. Unfortunately, 
effectiveness is not quite enough for achieving our goal. 

Definition 2.10. We say that G acts effectively on subsets ofM if, for any open subset 

U C M, 

{5eG| = forallpe[/} = {e}. 

We say that G acts locally effectively on subsets of M if, for any open subset U C M, 

{.g G G| g ■ p ~ p, for all p E Af } is a discrete subgroup of G. 

Observe that effectiveness on subsets implies effectiveness. The converse is not 
true in general, but it holds for all analytic group actions. 

Tlieorem 2.11. If a group G acts on a manifold V locally effectively on subsets, then 
for n e N big enough, the induced diagonal action ofG on '^"^ is locally free on on 
open and dense subset ofV^^'^\ This is equivalent to saying that the orbit dimension 
is equal to the dimension of G on this open and dense subset. We denote by hq the 
minimal integer for which this is true. 

So at least for all analytic group actions (moding out by a subgroup if necessary), 
acting (diagonally) on more and more copies of a manifold will eventually lead to a 
regular and locally free action on an open subset. A local moving frame will thus 
exist and provide us with the tools we need to obtain a complete fundamental set of 
invariants so completely characterize the orbits. 

We now explain how to construct a (local) a moving frame and to obtain a complete 
fundamental set of invariants. A more detailed exposition can be found in Chapter 
8]. 

Let g = {gi, . . . , gr) be local coordinates for G in a neighborhood of the identity. 
Suppose that G acts regularly on M. For simplicity, let us assume in addition that the 
orbits of G have the same dimension r as G itself. In other words, we are assuming that 
the action is locally free. A simple variation allowing us to deal with merely regular 
actions will also be explained shortly. 
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• Step 1: Write down the group transformation equations x = g ■ x explicitly. 

Xl = fi{gi,... ,gr,Xi,... ,Xm), 

< : 

^ Xm fmidl') • ' • 1 9riX\i - • - )^m)* 

• Step 2: Choose constants ci , . . . , cv e R and set r of the transformed coordi- 
nates equal to those constants. For simpUcity, we relabel the coordinates and 
write 

higi,--- ,9r,Xi,... ,Xm) = Ci, 

: (1) 

^ fr{gi7 ■ • ■ j5'r;^l5*** ^-^m) ~ ^r* 

These equations are called the normalization equations. 

• Step 3: Solve the normalization equations for g = (ffi, • • • ,5r)- The solution 
g = p{x) is a moving frame. 

• Step 4: Compute the action of the moving frame on the remaining coordinates. 
The set of resulting functions 

Xr+l\g=p(x) ^ Il{xi, . . . ,Xm)-, 

< : 

^rn\g=p[x) ~ Im-s{X\, . . . ,Xm)- 

is a complete fundamental set of local invariants. 

The choice of constants in Step 2 is somewhat arbitrary: we are free to choose any 
numbers for which a solution to the normalization equations exists, provided that these 
constants define a cross-section (i.e. provided that the normalization equations define 
a submanifold which is transversal to the orbits). To simplify the solving process, it is 
usually a good idea to choose as many constants as possible to be zero. 

If the action is not free but merely regular, we can still find a system of functionally 
independent local invariants. What we do is the following. Let s be the dimension of 
the orbits of G (s < r). We solve the s equations fi{g, x) = ci, . . . , ,fs{9, x) = Cg 
for s of the group parameters and replace them in the remaining equations Xs+i = 
fs+i{g, x), . . . , Xm = fm{g, x) to get the m — s invariants. The other group parame- 
ters will not appear in the equations. This procedure is called a partial moving frame 
normalization method. 

Equipped with these theoretical tools, the computation of invariants becomes a 
simple systematic procedure. We can thus feel free to consider any Lie group action 
imaginable and try to obtain its invariants. As we have seen, in theory, results are 
guaranteed provided that the group action is locally effective on subsets, which we can 
always arrange in the case of analytic group actions. 
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3 Application to 3D Shape Reconstruction 



Let us think for a moment about the process of taking a picture. This process involves, 
first of all, the placement of a camera in space. Then, particles of light starting from 
the shape (points in R"^) travel on a straight line in the direction of the camera center, 
leaving its trace on a film, i.e. on the intersection of the picture plane and the straight 
travel line. So to the picture-camera system placed somewhere in M'^, there corresponds 
a set of n straight lines in R'^ representing paths of light going from the object to the 
camera center 

This process can be seen as a group action generated by an action of SE{3) and an 
action of M" on the camera center together Pq with the image points Pi , . . . , P„. The 
idea is to allow each Pi to move independently along the line through Pi and Pq, while 
allowing the line configuration to be rotated and translated in space. As discussed in the 
introduction, including the camera center on the space acted on by the group will allow 
us to obtain significant equivalence classes. This is the key to guarantee the existence 
of non-trivial invariants and these invariants can be obtained by our systematic method. 

So, given is a 2D image depicting n points pi, . . . ,Pn G ■ We assume this 
picture was taken by a camera with fixed internal parameters. These parameters can be 
calibrated beforehand, so that the focal length is = l[] and the 2D image coordinates 
match the 3D coordinates as defined below. We embed the picture-camera system in 

by setting the camera center to be pg = (Oi 0, 0) and the picture points pj's to 
be Pi = Pi y. T . This is of course, in general, not the actual position in which the 
picture was taken. However, there exists a rigid transformation g e SE(Z) such that 
9 ■ (pojPi, . . . ,p„) = (*Poj?5ii ■ ■ • j?5n) corresponds to the actual position of the 
picture-camera system at the moment where the picture was taken. In other words, if 
the object is made of n points, say Oi , . . . , On, then each transformed image point *Pi 
lies on the straight line passing through Oi and the camera center *Po- In order to fuUy 
formulate the problem in terms of orbits, we want to consider smooth transformations 
that will map the image points . . . , *p„) to the object points (Oi, . . . , On). For 
this, we allow each point '^i to move independently along each ray of light so to go 
back to its source on the object. 

We would like to determine where *Po the O/s lie. Given a picture, it is of 
course impossible to determine the points (*Poj Oi, . . . , On)- However the "line ar- 
rangement" defined by the picture-camera system provides us with important informa- 
tion. In particular, we know that the points (*Po, Oi, . . . , On) lie on the orbit of the 
Lie group action on {{Pq, Pi, . . . , Pn) £ M.^ x (R^ )><(")} defined by: 



with R e 50(3) a rotation, T e R^ a translation and Aj e R, a factor of depth, 
for i — 1, . . . , n. Observe that the action of R" parameterized by the A's commutes 
with the action of SE{2) generated by the rotation R and the translation T. Therefore, 

'the value is arbitrary, it simply fixes tlie overall scale of the 3D reconstruction. 



Po 



RPo + T 

R{P, + X,{P^-Po))+T, foit = l,... ,n, 
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this defines a finite dimensional Lie group action, more precisely the action of a (6 + 
n)-dimensional Lie group on a (3n + 3)-dimensional manifold. Note that, although 
invariant-based techniques have already been used in the field for the general projective 
and affine transformation groups I plj , pj| , p^ , the Lie groups we shall define have little 
to do with these traditional transformation groups. 

Assuming that the picture points are distinct, then the group action is regular and 
the orbits are 6-dimen sion al, for n ~ 1, and (6 + n) -dimensional as soon as n > 2. 



Therefore by Theorem 2.3, there are 2n — 3 fundamental invariants whenever n > 2. 
We now follow the steps of the moving frame normalization method to obtain them. 
We set 

Pq = (0,0,0)^, 
Pi -(0,1,0) = 0, 

A -(0,0,1) = 0, 

P2- (0,0,1) = 0, 
and P,; • (1,0,0) = 1, for alH = 1, . . . , n. 



Solving for the group parameters, we obtain 



where / 



T = 
R = 

Ri = 



R2 = 

Rs = 

X^ = 

-J/l£2+£iy2 



-RPo, 
R1R2R3, 
/ 1 







a 



f 



V 











(2) 



Vl 







xi 





1. 



(/?.(P.-Po)). 
_ Z2{xi+yi)-zi{xix2+yiy2) 



(xi,yi,zi)^ = Pi - Po and 



{x2, 1/2, Z2) = P2 — Po- These group parameters define a moving frame (MF). Re- 
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placing the moving frame into the transformation equations, we get: 



IMF 



IMF 



IMF 



IMF 



(0,0,0f , 
(l,0,Of , 



1 



fy/xl+yf+zl(xiy2-X2Vi)+g[z2{xl+yl)-zi{xiX2+ViV2)] 
(xiX2+yiy2+ziZ2)\/ x\-\-y\\/ p+g^ 



/ , 1 \ 

f^/xl+yj+zl{xiyi-Xiyi)+glzi(xl+yl)-zi{xiXi+yiyi)] 
ixi Xi+yiyi+ziZi)y/xl+yfy/ P+g^ 



g\/xl+yf+zl{xiyi-xiyi)+f[zi{xf+yf)-zi{xiXi+yiyi)] 
{xixi+yiyi+zizi)-s/xf+y1y/ P+g^ 



for alH = 3 , . . . ,n, where {xi,yi, Zi) = Pi — Pq. Each component of these vectors is 
an invariant of the group action. 

Let us try to understand the geometric meaning of these expression. Observe that 

^ f2 _^ g2 — ||(a;i,j/i,zi)x(a;2,j/2,^2) 

above system as: 



\/xl+y'l+zl 



After a few manipulations, we can rewrite the 



IMF 



IMF 



IMF 



IMF 



(0,0, Of, 
(1,0, of 



1 



\{xi,yi,zi)x{x2,y2,Z2)\ 
(xi,yi,zi)-{x2,y2,Z2) 



/ 1 \ 

l{xi,yi,zi) x (xi,yi,Zi)] -l{xi,yi,zi) x (x2,y2-Z2)] 
l{xi,yi,zi)-{xi,yi,Zi)\ \\{xi,yi,zi)x{x2,y2,Z2)\\ 
1 ixi,yi,Zi)-[ix2,y2,Z2)x{xi,yi,zi)]\\{xi,yi,zi)\\ , 
\ [{xi,yi,zi)-{xi,yi,Zi)] ||(xi,yi,2i)x(x2,y2,Z2)|| / 



It now becomes clearer that the components of P2 \ and Pi \ are sine or co- 
sine of angles between the directions spanned by PiPq, P2P0, PiPo and the directions 
orthogonal to them. These are clearly invariant by translation, rotation, and motion 
along the projection hnes. As a fundamental set, we simply pick the only 2n — 3 
non-constant invariants: 

J ^ \\{xi,yi,zi) X (3:2,^2,^2)11 

{xi,yi,zi) ■ {x2,y2,Z2) 
J ^ [{xi,yi,zi) X {x,,y,,z,)] ■ [{xi,yi,zi) x (2:2, j/2, ^2)] 

[(2:1,^1, zi) • {xi,yi,Zi)] j|(a;i,i;i, zi) X {x2,y2,Z2)\\ 
J ^ {xt,yz,Zi) ■ [{x2,y2,Z2) x {xi,yi, zi)]\\{xi,yi, zi)\\ 

[{xi,yi,zx)-{xi,yi,Zi)] ||(a;i,yi, 2:1) x {x2,y2,Z2)\\ 
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for i = 3, . . . ,71. 

Each picture taken defines a point in x (R'^)^(") and therefore determines an 
orbit of our group action. Each orbit is characterized by the set of 2n— 3 equations given 
by the invariants. More precisely, indexing the pictures with the discrete parameter 
T = 1, . . . , i, we have 

/,(PJ,Pi,... ,P„) = a[, fori = 2,... ,n, 
J,(Po^ Pi, . . . , P„) = /?;, for J = 3, . . . , n. 

for appropriate constants a^'s and /3J's. These constants are prescribed by the pictures: 
since the picture-camera system itself belongs to the orbits, we have 

We are interested in solving the equations 

/,(<P5,Oi,... ,0„) = al, fori = 2,... ,n 
JJ(q35,Oi,... ,a) = /3J, forj = 3,... ,n. 

for T = 1, . . . , i. For 

We have (2n — 3)t (non-linear) equations with in + 3t unknowns, the solution of 
which is determined up to a rotation and translation of the 3D camera-object system as 
a whole, which can can fix arbitrarily, thus eliminating six variables^. For n > 3 and 
t > 2^Ig , the number of equation is greater than the number of unknown so we can 
try to solve them. 

Experiments with real video images have been performed (see Fig.[l| using a se- 
quential non-linear optimization technique based on the Levenberg-Marquardt algo- 
rithm |]Io|]. The feature points used in the pictures are endpoints of lines and rectangles 
associated from one image to the next with a tracking procedure [jl|]. The reconstructed 
3D object is valid in any view, even if the bottom and left side elements are not perfectly 
replaced, due to the tracking noise. The computations take only a few minutes. 

Observe that our camera-system does not take into account the angle of the camera; 
the orientation of the image plane was not used in our description of the camera-picture 
system. Our invariants are invariant under a rotation of the image plane and so we 
cannot use them to recover the camera orientation. But this has its advantages, as 
illustrated by the following lemma. 

Lemma 3.1. The motion of the camera between two pictures is a pure rotation (i.e. a 
rotation around the center of projection Pq) if and only if the values of the invariants 
{li, Jj\i ~ 2, . . . ,n, j — 3, . . . ,n} evaluated on any corresponding points in the two 
views are equal. 

^However, we should keep in mind that the choice of these variables will affect the numerical 
resolution||6||. 
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Figure 1 : A reconstruction example: a) the first of six images, with line and rectangle 
features, b) a similar view of the 3D reconstructed features, c) a top view, d) a side 
view of the reconstruction. 



Proof. Invariance of our invariants under pure rotations is obvious from the construc- 
tion of the invariants. 

To prove that equality of our invariants evaluated on all corresponding points guar- 
antees that the camera motion is a pure rotation, observe that the first invariant I2 is 
the tangent of the angle between the lines ^PoCi and ^oC2. Its value remains constant 
for fixed d, ©2 only if *Po moves along a circle around the O1O2 axis. This holds 
for all possible choices of Oi and O2 so the camera center must He somewhere on the 
intersection of a set of circles, which can be taken to intersect at merely one point to 
guarantee that the camera center does not move. □ 

4 Variations on the theme 

4.1 Taking the angle of the camera plane into account 

If we want to recover the camera rotation, we need to modify the system to introduce 
orientation information. Let Pl denote the lower left comer of the camera plane (em- 
bedded in K"*) and the upper left corner of the camera plane. These two points are 
sufficient to uniquely define the orientation of the camera plane. Consider the following 
Lie group action on {(Po, Pi, P^,Pi,... ,P„) G {W'Y^^^^ x (R^) ><(«)}; 

Po = PPo+T 

Pl = RPl+T 

= RP^ + T 

Pi = R{Pi + Xi{Pi-Po))+T,foTi = l,...,n, 

with R e 50(3) a rotation, T e a translation and Ai e M, a factor of depth, for 
i = 1, . . . , n. Again, this defines a finite dimensional Lie group action, more precisely 
the action of a (6 -|- n)-dimensional Lie group on a (3n -|- 9)-dimensional manifold. 
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Assuming that the points are distinct, this group action is regular and the orbits are 
6 + ri-dimensional, for any n. Therefore by Theorem 2.3, there are 2n + 3 fundamental 
invariants. 

We set 



Pl 
Pl 

pL 

and A 



Pa 

(0,1,0) 
(0,0,1) 
(0,0,1) 
(1,0,0) 



(0,0,0)^, 

0, 

0, 

0, 

1 , for all i = 1 , . 



For simplicity, we recycle the variables used for the moving frame in the previous 
case. Solving for the group parameters, we obtain a very similar result, with T, R 
and \i defined as in eq. (|), except that (xi, yi, zi)^ and (2:2, 1/2, Z2)^ replaced 
by (mi, wi, wi)-^ = Pl — Pa and (u2,W2, W2) = P^ ~ Pa respectively. These group 
parameters define a moving frame for the new transformation group. Replacing the 
moving frame into the transformation equations, we get a similar yet different result: 



Pl 



MF 



MF 



IMF 



IMF 



(0,0,0)^, 

(||Pi- Poll, 0,0)^, 

I (til, 1)1,11)1) 1 1 
\\(,Ul ,V1 ,Wl) X (U2 ,V2 ,W2)\\ 
|(til,t)l,U)l)|| 



1 

[(mi ,Vi,Wi)x (xi,yi,Zi)]-[(ui ,Vi,Wi)x (U2 ,f 2 ,t"2)] 



[{ui,vi,wiy{x 

{Xi,Vi,Zi)-[(u2 


,Vz 
V2 


,^.)] 

W2)X 


(«1 
("1 


vi 


U)l)x(«2 

(«i 


V2 
Vi 


W2) 1 

toi) 


l{ui,vi,wi)-(x 




,^^)] 


(«1 


vi 


Wi)x(u2 


V2 


W2) 1 



for alH = 3, . . . , n, where {xi, yi, Zi) — Pi — Pq. 

The new set of fundamental invariants for this group action is: 



II 
lo 

Jo 

h 

J, 



\\Pl-Po\\ 

(tt2, 1^2,^2) ■ (Ml,fl, Wl) 

||(Ml,'yi,Wi)|| 
\\{ui,Vi,Wi) X {U2,V2,W2)\\ 

||(ui,Wi,Wi)|| 
[{ui,Vi,Wi) X {xi,yi,Zi)] ■ [{ui,Vi,Wi) X {U2,V2,W2)] 

[{ui,vi,wi) ■ {xi,yi,Zi)] ||(ui,i;i, wi) x {u2,V2,W2)\\ 

{Xi,yi,Zi) ■ [{U2,V2,W2) X {ui,Vi,Wi)]\\{ui,Vi,Wi)\\ 

[{ui,vi,wi) ■ {xi,yi,Zi)] \\{ui,vi,wi) X (u2, 1^2, W2)|| 



for i — 1, . . . ,n. 
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Each picture taken defines a point in (M'') ^ x (M-*) ^ and therefore determines 
an orbit of our group action. Each orbit is characterized by the set of 2n + 3 equations 
given by the invariants. Given t pictures, we have to solve a system of (2n + 3)t 
equations with 3n + 9t unknowns, 6 of which can be fixed arbitrarily. Again for t and 
n big enough (n > 3 is necessary, t > 3 for n = 4, etc.), the number of equation 
is superior to the number of variables. We can search for a solution within the same 
optimization framework used in the previous case. The values of 'Pq, and ^p^"^ 
must be included in the solution. We thus obtain both camera position and orientation 
for each picture. 



4.2 Letting the focal distance vary 

It is slightly less trivial to set up the group transformation equations in the case were the 
focal length is allowed to vary from one image to the next. Consider Pm, the closest 
point to Pq on the image plane, i.e. the embedding of the middle point of the picture. 
Changing the focal distance corresponds to transforming Pm into a point P^ with a 
real parameter a according to the rule 

P'M^PM + a{PM-Po)- 

The induced action on the Pi 's can be taken as a rotation about the center of camera Pq 
which preserves the distance to the camera center. More precisely, each Pi is moved 
to a new point P/ in such a way that ||P/ — Po|| = |jPi — Po|| and that its transformed 
picture point is p- = pi + a{PM — Po)- Since the picture point is given by Pi = 

Po + ^p^_/^]7^pl_p^y we find that 

PI = P^ + \\p. - Po II (Pi-i^HPM-Po) + "(^A^ - 



\ ^p,.ji:uz-Po) +^(PM-Po)\\ 

Combining with translations of the Pj's along the line P^Pq together with rotations 
and translations of the line arrangement as a whole, we get the following (n + 7)- 
dimensional Lie group action: 

Po = RPo + T, 

Pm = R{PM + a{Po-PM))+T, 



R[Po + {l + Xi)\\Pi-Po\ 



iPi-Po)-(PM-Po) + "(-^M 



I (P<-Po)-(Pm-Po) 



+ a(Pj 



M 




Observe that the change of focal length parameterized by a conmiutes with the action 
of each Aj on Pj, because it preserves the norm || Pj — Po||. 

We apply the previous moving frame normalization technique by setting: 

Po = (0,0, of, 

Pm = (1,0, Of, 

Pi -(0,0,1) = 0, 

and Pi • (1,0,0) = 1, for alH = 1, . . . , n. 
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The corresponding group parameters are similar to eq. (Q) for R and T, except 

that {xi,yi,zi)'^ and {x2, 1/2, ^2)^ must be replaced by wi, wi)^ = Pm — Pq and 
{u2, V2,W2)'^ = Pi — Pq- The other parameters are: 

1 

- 1, 



\\Pm~Po\ 

IIP. -ftiiiift,- Poll + Jg7:^ 

Replacing these group parameters into the equations for the P, 's, we obtain the follow- 
ing complete fundamental set of invariants: 

. ll(PAf-Po) X (A-Po)ll 



(Pi - Po) • (Pm - Po) (1 + \\Pm - Poll - ||Pa/ - PoP) ' 

(Pm - Po) X (P ^ Po) ■ (Pm - Po) x (Pi -~ Pp) 

||(Pm - Po) X (Pi - Po)||(P - Po) ■ (Pm - Po) (1 + ||Pm - Poll - ||Pm - Po|P) ' 

J ^ (P - Po) ■ [(Pi - Po) X (Pm - Po)l ||Pm - Poll 

||(Pm - Po) X (Pi - Po)[|(P - Po) • (Pm - Po) (1 + |[Pm - Po|| - |[Pm - PolP) ' 

for i — 2, . . . ,n. 
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